A file system is that component of the operating system of a computer that breaks up the available storage area of some device (disk) into an arbitrary number of files, each of which appears, to its users, as if it were a contiguous storage device in its own right. However, such contiguity within a file is usually an illusion. Files are usually build up out of a series of smaller contiguous regions, called extents. To implement an illusion of contiguity, the file system maps any address given to it with respect to a file (file address) into the appropriate physical address on the disk where that part of the file actually resides.
Current art provides a wide variety of such mapping functions, also called file representation methods, most of which fall into two categories. The first, called extent list file representation, maintains a simple array, or a chain of such arrays, for each file. Each array element identifies the location and size of some extent on the disk. The range of file addresses that map to this extent is offset from the start of the file by the sizes of all the extents that are listed before it in the extent list. For example, consider a file having three extents: the first extent being one block long. Also assume the second four blocks long, and the last two blocks long, that a block is some atomic unit of disk storage space, say, 1000 bytes, and that file address 0 points to the first byte of the file. Then file addresses 0-999 map to the disk locations pointed to by the first extent of the file, file addresses 1000-4999 map to the disk locations pointed to by the second extent of the file, and file addresses 5000-6999 map to the disk locations pointed to by the third extent of the file.
The extent tree file representation, the second category of file representation methods, typically uses only a single extent size, say, one block. The enormous number of extents that results for large files is then managed by arranging them into some form of a tree structure. For example, in the popular UNIX(tm) operating system, a 13 element array is used as the root structure. The first ten elements of the array point to the first ten blocks of the file. The eleventh element points to a block which in turn contains pointers to the next 256 blocks in the file. The eleventh element is called, in UNIX terminology, a single level indirection pointer.
The twelfth element of the root array provides second level indirection. It points to a block, which in turn has pointers to 256 other blocks, each of which has pointers to a set of 256 data blocks of the file. Thus second level indirection maps up to 256.sup.2 =65536 data blocks into the file. This idea extends to the thirteenth root array element, which provides third level indirection, or access to an additional 256.sup.3 blocks, which is more than what can be supplied by today's disk technology.
A number of serious performance problems are known to occur with the extent tree file representation method. Random access into a medium or large scale file results in multiple I/O operations for each data block referenced, one for each pointer block that must be accessed to get at a data block. Another performance problem occurs when the potentially very low contiguity that is possible in such files is approached in practice. The extra head seeks necessary to sequentially access highly discontiguous files has been shown to slow down data transfers to and from a file up to a factor of six. In addition, opportunities to do multiblock transfers, thus also minimizing rotational latency as a performance degradation factor, occur less often with the extent tree file representation method. In fact, such contiguities are undependable to the point that no extent tree based implementation known to the applicant has taken advantage of them when they do occur.
The extent list file representation method has a different set of problems. All of them center on the fact that there has been no really effective way to select the sizes of the extents which are to go into a file. Typically, when a file is created, it is not obvious (to the file system at least) how much data will be written to it before it is closed. If the file system chooses a large size for the extents to be appended to the file, and only a little information is written out to the file, then a large amount of unused space is attached to the file which could be better used elsewhere (internal fragmentation). If, on the other hand, a small extent size is chosen and a large amount of data is written out, then the file is potentially highly discontiguous, and the resulting large number of extents will not even have the efficiencies of a tree structure to keep them organized. For those files which are created manually and whose final size is approximately known one method to minimize the tradeoff is to provide an initial size and an extent increment size parameter to the user at the time those files are created.
For automatically created files, and for user created files whose final size can not easily be guessed, the typical system will default the initial size to zero and selects a large extent increment size, usually one track, and simply accepts the high internal fragmentation that results as simply unavoidable. A variation on the above method, such as freeing up the excess space in a file when it is closed, promotes the creation of small, relatively unusuable extents on the disk (external fragmentation), whose magnitude is approximately equal to the internal fragmentation it replaces. When the external fragmentation is in turn minimized by permitting the (manual or automatically made) extent requests to be automatically broken up into a series of smaller extents as required, then the contiguity within a file becomes unpredictable, and often approaches the worst case, namely one block per extent.